fix: upstream istio support by juliusvonkohout · Pull Request #3021 · kubeflow/trainer

juliusvonkohout · 2025-12-05T19:58:52Z

@andreyvelich may you also patch it in helm for the jobset ?

github-actions · 2025-12-05T19:59:01Z

🎉 Welcome to the Kubeflow Trainer! 🎉

Thanks for opening your first PR! We're happy to have you as part of our community 🚀

Here's what happens next:

If you haven't already, please check out our Contributing Guide for repo-specific guidelines and the Kubeflow Contributor Guide for general community standards.
Our team will review your PR soon! cc @kubeflow/kubeflow-trainer-team

Join the community:

Slack: Join our #kubeflow-trainer Slack channel.
Meetings: Attend the Kubeflow AutoML and Training Working Group bi-weekly meetings.

Feel free to ask questions in the comments if you need any help or clarification!
Thanks again for contributing to Kubeflow! 🙏

coveralls · 2025-12-05T20:03:26Z

Pull Request Test Coverage Report for Build 21665617792

Details

0 of 0 changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 51.217%

Totals
Change from base Build 21610377640:	0.0%
Covered Lines:	1241
Relevant Lines:	2423

💛 - Coveralls

charts/kubeflow-trainer/templates/manager/deployment.yaml

manifests/base/manager/manager.yaml

charts/kubeflow-trainer/templates/manager/deployment.yaml

juliusvonkohout · 2025-12-16T13:15:57Z

/retest

Copilot

Pull request overview

This PR adds Istio sidecar exclusion configuration and seccomp security profiles to improve compatibility with Pod Security Standards (PSS) restricted policies when using Istio service mesh. The changes ensure that webhook traffic on port 9443 bypasses the Istio sidecar and adds RuntimeDefault seccomp profiles for enhanced security.

Key Changes:

Added Istio traffic exclusion annotation for inbound port 9443 to prevent sidecar interference with webhook communication
Configured RuntimeDefault seccomp profiles at the pod security context level for compliance with restricted PSS

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
manifests/third-party/jobset/kustomization.yaml	Adds kustomize patch for jobset-controller-manager deployment with Istio annotation and seccomp profile
manifests/base/manager/manager.yaml	Updates trainer manager deployment with Istio exclusion annotation and seccomp security context
charts/kubeflow-trainer/templates/manager/deployment.yaml	Adds Istio traffic exclusion annotation to Helm chart template for trainer manager

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

juliusvonkohout · 2025-12-30T15:23:38Z

@andreyvelich i am waiting for this to merge kubeflow/manifests#3314
/lgtm from my side if the tests are green.
Maybe you have to rebase the branch.

andreyvelich

/lgtm
@juliusvonkohout Can you rebase this PR to fix CI please ?
/cc @tenzen-y @astefanutti

Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

Add security context and annotations for Istio traffic management. Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

Added a patch to modify the jobset-controller-manager deployment annotations and security context. Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

Removed security context seccomp profile from deployment. Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

Removed security context seccompProfile configuration. Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

Removed security context seccompProfile from spec. Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

google-oss-prow · 2026-02-04T09:17:11Z

New changes are detected. LGTM label has been removed.

google-oss-prow · 2026-02-04T09:17:14Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from andreyvelich. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

juliusvonkohout · 2026-02-04T09:18:49Z

@andreyvelich rebase is done

andreyvelich · 2026-02-05T03:42:25Z

manifests/base/manager/manager.yaml

+      annotations:
+        traffic.sidecar.istio.io/excludeInboundPorts: "9443"


Sorry for the delay @juliusvonkohout!

We discussed about it with @kannon92 on JobSet issue: kubernetes-sigs/jobset#1052 (comment)

Since other platforms might not use Istio (e.g. Cilium, Linkerd, etc.), I would suggest to add custom patch to the kubeflow-platform overlay to append this annotation alongside JobSet change:

trainer/manifests/third-party/jobset/kustomization.yaml

Lines 20 to 30 in 6ec7112

patch: |-

apiVersion: apps/v1

kind: Deployment

metadata:

name: jobset-controller-manager

namespace: kubeflow-system

spec:

template:

metadata:

annotations:

traffic.sidecar.istio.io/excludeInboundPorts: "9443"

cc @kubeflow/kubeflow-trainer-team

From Slack:

I don't suggest you to do that in kubeflow/manifests
Similar to JobSet, just update kubeflow-platform Kustomize overlay in kubeflow/trainer repo to add patch for kubeflow-trainer-controller-manager deployment:

trainer/manifests/third-party/jobset/kustomization.yaml

Lines 15 to 30 in 6ec7112

- target:

group: apps

version: v1

kind: Deployment

name: jobset-controller-manager

patch: |-

apiVersion: apps/v1

kind: Deployment

metadata:

name: jobset-controller-manager

namespace: kubeflow-system

spec:

template:

metadata:

annotations:

traffic.sidecar.istio.io/excludeInboundPorts: "9443"

E.g. you can do that here: https://github.com/kubeflow/trainer/blob/6ec7112a3b397d0b160330f78f9b6bbcd4c59a0f/manifests/overlays/kubeflow-platform/kustomization.yaml
For the Helm Charts, simple use this Value to configure appropriate label when you do helm install
{{- include "trainer.manager.selectorLabels" . | nindent 8 }}

trainer/charts/kubeflow-trainer/templates/manager/deployment.yaml

Line 31 in 6ec7112

labels:

helm install kubeflow-trainer oci://ghcr.io/kubeflow/charts/kubeflow-trainer --version 2.1.0 --set trainer.manager.selectorLabels traffic.sidecar.istio.io/excludeInboundPorts=9443

Maybe a GSOC applicant can fork my branch and do the changes :-) and raise a PR against my the branch here. I will ask https://cloud-native.slack.com/archives/C0742LBR5BM/p1770634485333769

juliusvonkohout · 2026-02-12T12:21:50Z

Might be continued in #3189

andreyvelich · 2026-02-19T00:37:02Z

This PR is fixed by: #3189
/close

google-oss-prow · 2026-02-19T00:37:07Z

@andreyvelich: Closed this PR.

Details

In response to this:

This PR is fixed by: #3189
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

google-oss-prow bot requested review from jinchihe and kuizhiqing December 5, 2025 19:58

google-oss-prow bot added size/XS size/S and removed size/XS labels Dec 5, 2025

google-oss-prow bot added size/M and removed size/S labels Dec 5, 2025

juliusvonkohout changed the title ~~pss-restricted-fixes~~ fix: pss-restricted-with-istio Dec 5, 2025

andreyvelich reviewed Dec 8, 2025

View reviewed changes

charts/kubeflow-trainer/templates/manager/deployment.yaml Show resolved Hide resolved

manifests/base/manager/manager.yaml Outdated Show resolved Hide resolved

juliusvonkohout commented Dec 10, 2025

View reviewed changes

charts/kubeflow-trainer/templates/manager/deployment.yaml Outdated Show resolved Hide resolved

juliusvonkohout assigned juliusvonkohout and andreyvelich Dec 16, 2025

Copilot AI review requested due to automatic review settings December 30, 2025 15:17

Copilot AI reviewed Dec 30, 2025

View reviewed changes

kubeflow deleted a comment from Copilot AI Dec 30, 2025

juliusvonkohout mentioned this pull request Dec 30, 2025

cleanup trainer overlay kubeflow/manifests#3314

Open

andreyvelich reviewed Jan 3, 2026

View reviewed changes

google-oss-prow bot requested review from astefanutti and tenzen-y January 3, 2026 20:12

google-oss-prow bot added the lgtm label Jan 3, 2026

juliusvonkohout mentioned this pull request Jan 12, 2026

Fix trainer validator webhook call failure kubeflow/manifests#3322

Closed

andreyvelich mentioned this pull request Jan 12, 2026

Wishlist for v0.11.0 kubernetes-sigs/jobset#1052

Open

juliusvonkohout added 4 commits February 4, 2026 10:15

pss-restricted-fixes

9a0850b

Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

Enhance deployment.yaml with security context and annotations

dcd83f9

Add security context and annotations for Istio traffic management. Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

Update kustomization.yaml with new patches

9fdcccb

Added a patch to modify the jobset-controller-manager deployment annotations and security context. Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

Remove seccomp profile from manager deployment

db3fa5c

Removed security context seccomp profile from deployment. Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

juliusvonkohout added 2 commits February 4, 2026 10:16

Remove seccompProfile from security context

92513eb

Removed security context seccompProfile configuration. Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

Remove seccompProfile from manager security context

6ec7112

Removed security context seccompProfile from spec. Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

juliusvonkohout force-pushed the pss-restricted-fixes branch from a2b6a53 to 6ec7112 Compare February 4, 2026 09:17

google-oss-prow bot removed the lgtm label Feb 4, 2026

andreyvelich reviewed Feb 5, 2026

View reviewed changes

sameerdattav mentioned this pull request Feb 9, 2026

chore: upstream istio support #3189

Open

juliusvonkohout changed the title ~~fix: pss-restricted-with-istio~~ fix: upstream istio support Feb 12, 2026

google-oss-prow bot closed this Feb 19, 2026

andreyvelich deleted the pss-restricted-fixes branch February 19, 2026 00:37

andreyvelich restored the pss-restricted-fixes branch February 19, 2026 00:37

juliusvonkohout deleted the pss-restricted-fixes branch February 19, 2026 08:40

		annotations:
		traffic.sidecar.istio.io/excludeInboundPorts: "9443"

	patch: \|-
	apiVersion: apps/v1
	kind: Deployment
	metadata:
	name: jobset-controller-manager
	namespace: kubeflow-system
	spec:
	template:
	metadata:
	annotations:
	traffic.sidecar.istio.io/excludeInboundPorts: "9443"

	- target:
	group: apps
	version: v1
	kind: Deployment
	name: jobset-controller-manager
	patch: \|-
	apiVersion: apps/v1
	kind: Deployment
	metadata:
	name: jobset-controller-manager
	namespace: kubeflow-system
	spec:
	template:
	metadata:
	annotations:
	traffic.sidecar.istio.io/excludeInboundPorts: "9443"

Comments

Conversation

juliusvonkohout commented Dec 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 5, 2025

Uh oh!

coveralls commented Dec 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 21665617792

Details

💛 - Coveralls

Uh oh!

Uh oh!

Uh oh!

Uh oh!

juliusvonkohout commented Dec 16, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

juliusvonkohout commented Dec 30, 2025

Uh oh!

andreyvelich left a comment

Choose a reason for hiding this comment

Uh oh!

google-oss-prow bot commented Feb 4, 2026

Uh oh!

google-oss-prow bot commented Feb 4, 2026

Uh oh!

juliusvonkohout commented Feb 4, 2026

Uh oh!

andreyvelich Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juliusvonkohout Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juliusvonkohout commented Feb 12, 2026

Uh oh!

andreyvelich commented Feb 19, 2026

Uh oh!

google-oss-prow bot commented Feb 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

juliusvonkohout commented Dec 5, 2025 •

edited

Loading

coveralls commented Dec 5, 2025 •

edited

Loading

andreyvelich Feb 5, 2026 •

edited

Loading

juliusvonkohout Feb 9, 2026 •

edited

Loading